Learning to Find Context Based Spelling Errors

A context-based spelling error is a spelling or typing error that turns an intended word into another word of the language. For example, the intended word “sight” might become the word “site.” A spell checker cannot identify such an error. In the English language— the case of interest here—a syntax checker may also fail to catch such an error since, among other reasons, the parts-of-speech of an erroneous word may permit an acceptable parsing. This chapter presents an effective method called Ltest for identifying the majority of context-based spelling errors. Ltest learns from prior, correct text how context-based spelling errors may manifest themselves, by purposely introducing such errors and analyzing the resulting text using a data mining algorithm. The output of this learning step consists of a collection of logic formulas that in some sense represent knowledge about possible context-based spelling errors. When, subsequently, testing text is examined for context-based spelling errors, the logic formulas and a portion of the prior text are used to analyze the case at hand and to pinpoint likely errors. Ltest has been added to an existing software system for spell and syntax checking. We have conducted tests involving mathematical, technical, and general texts. On the average, Ltest found 68% of context-based spelling errors in large texts and 87% of such errors in small texts. These detection rates are relative to words for which training was possible using the prior text. On the other hand, the number of false-positive diagnoses was small, involving on average 23 word instances (= 0.7% of the possible error instances) of a large text and 1 word instance (= 8% of the possible error instances) of a small text. These statistics indicate that the method is effective for the recognition of the majority of context-based spelling errors considered in the experimental tests.